The present invention concerns a method and device for processing components in an image. The invention further relates to a method and a device for encoding an image, and to a method and device for decoding an image.
At least one embodiment of the invention is applicable in the field of intra coding of chroma samples of video data.
Video data is typically composed of a series of still images or frames which are shown rapidly in succession as a video sequence to give the visual impression of a moving image. Video applications are continuously moving towards improved image resolution (greater number of pixels per frame, higher frame rate, higher bit-depth . . . ). A large quantity of video content is distributed in digital form via broadcast channels, digital networks and packaged media, with a continuous evolution towards improved quality and resolution (e.g. higher number of pixels per frame, higher frame rate, higher bit-depth or extended colour gamut). This evolution in technology puts increased pressure on distribution networks that already face difficulties in providing HDTV (high definition television) resolution and data rates economically to the end user. Consequently, further increases in data rate will put additional pressure on such distribution networks. To address this challenge, ITU-T (International Telecommunications Union, telecommunications Standardization Sector) and ISO/MPEG decided to launch a new video coding standard project in January 2010, known as High Efficiency Video Coding (HEVC). It will be appreciated that in what follows the term “HEVC” is used to represent the current implementation of HEVC, which is in course of standardization and will be subject to evolution.
HEVC codec design is similar to that of most previous so-called block-based hybrid transform codecs such as H.263, H.264, MPEG-1, MPEG-2, MPEG-4, SVC. Video compression algorithms, such as those standardized by the standardization bodies ITU, ISO and SMPTE, use spatial and temporal redundancies of images in order to generate data bit streams of reduced size. Spatial redundancy represents the mutual correlation between adjacent image pixels, while temporal redundancy represents the correlation between images of sequential images. Such compression processes make the transmission and/or storage of video sequences more effective.
According to HEVC an original video sequence comprises a succession of digital images which may be represented by one or more matrices the coefficients of which represent pixels. The images can be divided into slices where a slice can form part of the image or the entire image. In HEVC the slices are divided into non-overlapping Largest Coding Units (LCUs) which are generally blocks of size 64 pixels×64 pixels. Each LCU may in its turn be iteratively divided into smaller variable size Coding Units using a quadtree decomposition.
During video compression in HEVC, each block of an image being processed is predicted spatially by an “Intra” predictor (so-called “Intra” coding mode), or temporally by an “Inter” predictor (so-called “Inter” coding mode). Each predictor is a block of pixels obtained from the same image or another image, from which a difference block (or “residual”) is derived. In the Intra coding mode the predictor (Intra predictor) used for the current block (prediction block) is a block of pixels constructed from the information already encoded of the current image.
In HEVC, the coding order of the blocks of an image follows a kind of zigzag pattern, as illustrated in FIGS. 2A and 2B: a Largest Coding Unit (or LCU) is a block of 64×64 pixels recursively split into smaller blocks (16×16), (8×8) down to 4×4 sized blocks. FIG. 2A illustrates the non-uniformity of the splitting (which is signaled in the bitstream). As a result of this coding order, the 4×4 blocks of an image are indexed according to the numbering illustrated in FIG. 2B. Note that in FIG. 2B, numbers above and on the left of the figure represent respectively the index of lines and columns of pixels.
More illustratively, the scanning seen on FIG. 2B can be read as:                1. Process 16×16 sized blocks of indexes 0, 16, 32 and 48 in that order; Note here that the index number of a 16×16 block is the index number of the 8×8 block located in the upper left corner of the 16×16 block;        2. Process 8×8 sized blocks of indexes 64, 80, 96 and 112 (again in that order);        3. Process 16×16 sized blocks of indexes 128, 144, 160 and 176;        4. Finally, process the remaining 32×32 sized block of index 192        
For any N×N block spatial prediction methods can access 2N pixels (samples) on the top and left borders. Thus, for instance, the 16×16 block of index 0 would have access to pixels in columns 0 to 31 on its top border and in lines 0 to 31 on its left border (which may or may not exist if the block lies on the top and/or left outer borders of the image), and the 8×8 block of index 116 (whose first pixel coordinates is (56,16)) would access to line of pixels from pixel (56,15) to pixel (71,15) above and row from pixel (55,16) to pixel (55,31) on the left, effectively outreaching pixels not yet coded or even not existing.
To harmonize the prediction behavior, a coding method instead of directly accessing the image buffer can access a “prediction buffer”, where missing pixels are replaced by previous existing ones. For example, in the case of the 8×8 block of index 116 in FIG. 2B, the values of missing above pixels from pixel (64,15) to pixel (71,15) are replaced by the value of above existing pixel (16,63), while values of missing left from pixel (55, 24) to pixel (55,31) are set to the value of left pixel (23,56).
It may also be noted that blocks that are last in the Z shape ordering always have their outer border entirely filled by the padding process (i.e. no actual border pixel data exists there). Those are:                For 4×4 blocks: indexes 3, 7, 11, 15, 19, 23 . . . (i.e. index modulo 3 equals 3);        For 8×8 blocks: 12, 28, 44, 60 (i.e. index/4 modulo 3 equals 3);        For 16×16 blocks: 48, 112, 176, 244 (i.e. index/16 modulo 3 equals 3).        
An image of the video data to be transmitted may be provided as a set of two-dimensional arrays (also known as colour channels) of sample values, each entry of which represents the intensity of a colour component such as a measure of luma brightness and chroma colour deviations from neutral grayscale colour toward blue or red (YUV) or as a measure of red, green, or blue light component intensity (RGB).
A YUV model generally defines a colour space in terms of one luma (Y) and two chrominance (UV) components. Generally Y stands for the luma component (the brightness) and U and V are the chrominance (colour) or chroma components.
Among the various technologies in use in HEVC codec is the luma-based chroma prediction. Luma-based chroma prediction uses a linear model in a block to link luma components to chroma components. The parameters of the model may be determined on the fly, using samples located on the outer neighbouring borders of the block to be encoded and a least-mean square method.
The following model is given as example for generating an estimation of, i.e. predicting, the variable Y:
                              Y          ^                =                              ∑                          i              =              1                        N                    ⁢                                    a              i                        ⁢                          X              i                                                          (        1        )            where:                the Xi values are input to the prediction model, for instance from other signals or previous values of the Y signal;        the ai factors, ranging from 1 to N, are parameters (“weights”) for each input;        in the case for example where X0=1, a0 is a constant term        
In order to determine the model parameters, the quadratic error of the model for the Y signal over the pixels, indexed with n is expressed as:
                              e          C                =                              ∑            n                    ⁢                                    (                                                Y                  n                                -                                                      ∑                                          i                      =                      1                                        N                                    ⁢                                                            a                      i                                        ⁢                                          X                      i                                                                                  )                        2                                              (        2        )            
The least mean square method tries to determine the ai parameters minimizing this error value. The possible minimum where all derivatives with respect to said parameters are null may be considered as:
                              ∀          ⅈ                ,                                            ∂                              e                C                                                    ∂                              a                i                                              =                                    0              ⇒                                                ∑                  n                                ⁢                                                      X                    i                                    ⁡                                      (                                                                  Y                        n                                            -                                                                        ∑                                                      i                            =                            1                                                    N                                                ⁢                                                                              a                            i                                                    ⁢                                                      X                            i                                                                                                                )                                                                        =            0                                              (        3        )            
This can be rewritten as follows, showing that a classical system of N linear equations with N unknowns, to solve, is obtained:
                              ∀          ⅈ                ,                                            ∑              n                        ⁢                                          a                i                            ⁢                                                ∑                                      k                    =                    1                                    N                                ⁢                                                      X                                          k                      ,                      n                                                        ⁢                                      X                                          i                      ,                      n                                                                                                    =                                    ∑              n                        ⁢                                          X                                  i                  ,                  n                                            ⁢                              Y                n                                                                        (        4        )            
That can be expressed as a matrix system which is symmetrical, and on which known matrix solving algorithms such as the Gauss pivot can be used. The ai parameters may therefore be determined.
When coding a chroma block as intra (i.e. the chroma block depends on previously coded/decoded data from the current image), particular prediction modes can be applied, e.g. a directional bilinear prediction or a prediction by an average value. In all cases, samples the outer borders of the chroma block which may be referred to as a Coding Unit (hereafter CU), located inside of previously decoded CUs, are used. Those borders therefore contain decoded pixel data (samples), to which particular filtering might have been applied.
The prediction mode for predicting a chroma block in this example relies on an affine model:Cpred[x,y]=α·L′[x,y]+β  (5)
Here, the values of the prediction block made up of the pixels C[x,y] are linearly derived from the virtual luma values L′[x,y] which are in turn derived from the collocated decoded luma values.
The model parameters α and β are found using samples of the outer borders of the chroma block and the outer borders of the co-located luma block and a Least Mean Square method as presented above.
With reference to FIG. 1, the model parameters α and β are determined using encoded/decoded chroma samples on the top border 102 and left border 101 of the chroma block and co-located encoded/decoded luma samples from the top border 105 and left border 104 lying on the outside of the co-located luma block 106. The signals used as inputs are:                For the computation of the model parameters α and β:                    L′ values on left border 104 (x=−20) are obtained through filtering the decoded RecL value located on the column immediately on the left with the vertical convolution filter 107, resulting in the output: where Rec refers to reconstructedRecL′[x,y]=(RecL[2x,2y]+RecL[2x,2y+1])>>1  (6)            Note that (>>x) represents an integer division by 2x.            L′ values on top border 105 (y=−20) are filtered using the horizontal convolution filter 108, resulting in the output:RecL′[x,y]=(RecL[2x−1,2y]+2RecL[2x,2y]+RecL[2x+1,2y]+2)>>2  (7)                        For the values 106 used in the prediction, the filter 107 described for the left border is reused but on a different column:RecL′[x,y]=(RecL[2x,2y]+RecL[2x,2y+1])>>1  (8)        
Note that in a classical YUV4:2:0 colour space, the nominal position of the chroma sample inside a 2×2 luma block is in the middle of the left luma column, corresponding to the 2x horizontal position in the above formulas.
For such a simple model, the solution can be easily expressed analytically as:
                    α        =                                                            N                ⁢                                                      ∑                                          i                      =                      1                                        N                                    ⁢                                                            C                      i                                        ⁢                                          L                      i                      ′                                                                                  -                                                ∑                                      i                    =                    1                                    N                                ⁢                                                      C                    i                                    ⁢                                                            ∑                                              i                        =                        1                                            N                                        ⁢                                          L                      i                      ′                                                                                                                          N                ⁢                                                      ∑                                          i                      =                      1                                        N                                    ⁢                                      L                    i                                          ′                      ⁢                                                                                          ⁢                      2                                                                                  -                                                (                                                            ∑                                              i                        =                        1                                            N                                        ⁢                                          L                      i                      ′                                                        ⁢                                                                          )                                2                                              =                                                                      N                  ·                  XY                                -                                  X                  ·                  Y                                                                              N                  ·                  XX                                -                                  X                  ·                  X                                                      =                                          A                1                                            A                2                                                                        (        9        )            with N being the number of input samples in the borders and the sums simplified as XY, X, Y and XX X corresponding to
  N  ⁢            ∑              i        =        1            N        ⁢          L      i      ′      and Y corresponding to
  N  ⁢            ∑              i        =        1            N        ⁢          C      i      XY corresponding to
            ∑              i        =        1            N        ⁢                  C        i            ⁢              L        i        ′              ,XX corresponding to
            ∑              i        =        1            N        ⁢          L      i              ′        ⁢                                  ⁢        2              ,(notations that are used in what follows). Identically, we have for the constant parameter:
                    β        =                                                                              ∑                                      i                    =                    1                                    N                                ⁢                                  C                  i                                            -                              α                ⁢                                                      ∑                                          i                      =                      1                                        N                                    ⁢                                      L                    i                    ′                                                                        N                    =                                    Y              -                              α                ·                X                                      N                                              (        10        )            
However, determining the model parameters is a complex task and adds complexity particularly in the case the decoder.
Although the use of multiple LM modes has been studied to improve coding efficiency, its use in some cases leads to increased complexity, since using N modes would multiply the most complex part of the encoder by N.
The present invention has been devised to address one or more of the foregoing concerns.